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The impact that financial education had on the financial behaviors of (a) the agency staff who were trained to 
deliver the program and (b) the low-income individuals who participated in the program was investigated. Spe¬ 
cifically, the researchers examined the relationship between total number of financial education lessons com¬ 
pleted, prior financial experience, and improvement in individuals ’financial behaviors. The results provide 
some evidence that financial education may result in improved financial behaviors. However, the findings sug¬ 
gest that prior level offinancial experience may matter more than the number of lessons completed. Researchers 
may want to re-examine the indicators currently being used to show program impact and whether financial 
knowledge is the appropriate catalyst to foster behavior change. 
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Introduction 

High levels of consumer debt, low personal saving rates, 
and increases in personal bankruptcy filings have gener¬ 
ated concern that consumers lack adequate financial skills 
(Bell & Lerman, 2005; Lyons, Palmer, Jayaratne, & 
Scherpf, 2006; National Endowment for Financial Educa¬ 
tion, 2004). As the financial system has grown rapidly 
more complex, consumers have had to become more 
actively involved in managing their finances. Yet, many 
consumers, even those who would describe themselves as 
“financially savvy,” are having difficulty assessing their 
options and making sound financial decisions. The burden 
for low-income and disadvantaged individuals can be 
particularly overwhelming. In the current financial envi¬ 
ronment, it is easy for low-income and disadvantaged 
populations to fall victim to predatory lenders and finan¬ 
cial scams, especially because many lack adequate finan¬ 
cial education (Lyons & Scherpf, 2004). Basic financial 
management skills are important for all households, but 
they are particularly critical for low-income households to 
ensure long-term financial security. 

A number of financial education programs have been 
developed in recent years to address the financial educa¬ 


tion needs of low-income populations. However, research 
measuring the effectiveness of these programs has not kept 
pace. There are a number of reasons why limited research 
is available. First, researchers face challenges in collecting 
data from program participants with low financial literacy 
levels. Low-income participants are often difficult to track 
and have high program drop out rates and low survey 
response rates (Anderson, Zhan, & Scott, 2004; Lyons, 
2005; Lyons et al., 2006; Lyons & Scherpf, 2004). These 
factors, coupled with their reluctance to divulge personal 
information, limit the amount and type of information that 
can be collected. The result is that survey instruments are 
often kept short and simple to increase response rates and 
reduce measurement error. 

Second, researchers face challenges collecting impact data 
because of the nature of the organizations that deliver 
financial education programs. Most programs that target 
low-income populations are operated by small non-profit 
organizations with limited staff and financial resources. 
Relative to their operating expenses, program evaluations 
can be expensive to conduct, especially rigorous evalua¬ 
tions that use control groups and have a longitudinal com¬ 
ponent. In addition, many of the agency staff and volun- 
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teers do not have expertise in evaluation and lack the 
understanding and knowledge about how to measure 
program impact to show that their programs are working 
(Lyons, 2005; Lyons et al., 2006). In the end, there are few 
incentives for instructors to collect data and for partici¬ 
pants to provide information. 

Most organizations currently conducting program evalua¬ 
tions use only a post-test or a pre- and post-test to collect 
impact data (Lyons et al., 2006). However, given the 
nature of low-income populations and the constraints 
facing many organizations, it may be more efficient and 
effective to collect impact data using a retrospective pre¬ 
test (RPT). The RPT is administered in the same way as a 
post-test in that participants are asked to answer questions 
about their level of knowledge and behavior after the 
program. They are then asked to think back to their level 
of knowledge and behavior prior to the program. Despite 
the potential limitations of this method, RPT can reduce 
response-shift biases and provide a feasible and efficient 
evaluation method for collecting impact data (Lamb, 2005; 
Lamb & Tschillard, 2005). 1 

For the current study, a RPT was used to collect 4 years of 
repeated cross-sectional evaluation data from a nationally 
recognized financial education program. The data were 
used to investigate the impact that education had on the 
financial behaviors of the agency staff that were trained to 
deliver the program and of the low-income individuals 
who participated in the program. Controlling for prior 
financial behaviors, we estimated a series of probit models 
to determine if the amount of financial education received 
had an impact on (a) overall financial behavior and (b) 
anticipated changes in five specific financial behaviors. 

The “treatment effect” in our models was defined by the 
variation in the intensity of the treatment (i.e., the number 
of lessons completed) instead of by whether an individual 
received the treatment or did not receive the treatment (i.e., 
participated in the program or did not participate). Al¬ 
though this was not a traditional control group study, we 
were able to examine the impact of the education condi¬ 
tional on some level of program participation by compar¬ 
ing the behavior changes of participants who received 
more lessons to those who received fewer lessons. 

The findings presented provide insight into how financial 
education programs for low-income populations can be 
improved, especially with respect to program length and 
the indicators currently being used to show how financial 
education can translate into actual and anticipated behavior 
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changes. The results also provide insight into the impor¬ 
tance of controlling for previous financial experience. 
Finally, the findings show how RPTs can provide useful 
insight into the effectiveness of financial education pro¬ 
grams for low-income populations, underscoring the value 
of more traditional types of evaluation methods. 

Review of Literature 

Recent literature has provided general insight into the link 
between financial education and behavior change. Re¬ 
searchers typically have concluded that financial education 
results in positive behavior change. The majority of these 
studies have focused on collecting data from target popula¬ 
tions that are readily available and willing to participate in 
formal evaluations, such as employees, students, and 
financial counseling clients. For an overview of the litera¬ 
ture, see Bell and Lerman (2005), Braunstein and Welch 
(2002), Fox, Bartholomae, and Lee (2005), Hilgert, 
Hogarth, and Beverly (2003), Hogarth (2002), Hogarth, 
Beverly, and Hilgert (2003), Lyons (2005), Lyons et al. 
(2006), and National Endowment for Financial Education 
(2004). 

Studies that have concentrated on the effect of financial 
education in the workplace have focused on increasing 
employees’ savings and enhancing worker productivity. 
Bayer, Bernheim, and Scholz (1996) and Bernheim and 
Garrett (2003) found that employer-provided financial 
education increased employee participation in retirement 
plans and the amount saved for retirement and for general 
purposes. They also found that the effect of workplace 
financial education tended to be strongest for employees 
who saved little before the program. In another study, 
Garman, Kim, Kratzer, Brunson, and Joo (1999) found that 
75% of individuals who chose to participate in employer- 
sponsored financial education workshops felt more confi¬ 
dent in their ability to make investment decisions, and in 
turn, made better financial decisions following the work¬ 
shops. Kim and Garman (2003) also found that employer- 
provided financial education increased the financial confi¬ 
dence of program participants and resulted in improved 
financial practices and worker productivity. 

Studies that have focused on youth provide evidence that 
formal courses in personal finance increase financial 
knowledge and result in more positive financial behaviors. 
For example, Boyce and Danes (2004) found that a formal 
financial planning program had a significant and positive 
impact on high school students’ spending habits, savings 
behaviors, and confidence levels in managing money, even 
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3 months after they completed the program. Bernheim, 
Garrett, and Maki (2001) found that mandated financial 
education during high school resulted in higher savings 
rates and higher net worth when students reached adult¬ 
hood. 

Other studies have shown that financial counseling results 
in improved financial behaviors. Staten, Elliehausen, and 
Lundquist (2002) tracked credit counseling clients for 3 
years and found that those who received counseling were 
able to reduce their debt, improve their credit card man¬ 
agement, and lower their delinquency rates more than 
those who did not receive counseling. Hirad and Zorn 
(2001) found that borrowers who participated in pre¬ 
purchase homeownership counseling had a 19% lower 90- 
day delinquency rate than those who did not receive coun¬ 
seling. Kim, Garman, and Sorhaindo (2003) provided 
evidence that suggested that credit counseling may have 
direct effects on financial stressor events and indirect 
effects on individuals’ perceived financial well-being 
and health. 

Research that has measured the effectiveness of financial 
education for low-income populations has been more 
limited. Existing literature has been tied frequently to 
participation in Individual Development Account (IDA) 
programs. The goal of IDA programs has been to increase 
savings rates for low-income families by providing match¬ 
ing funds for savings toward a specific purpose such as 
homeownership, higher education, or to start a small 
business. Many of these programs have included a finan¬ 
cial education component. Research from one IDA pro¬ 
gram focused on knowledge gain and examined the levels 
of pre-training financial knowledge of program partici¬ 
pants (Anderson et al., 2004; Zhan, Anderson, & Scott, in 
press). The goal of the current research was to identify 
gaps in financial knowledge and to determine the financial 
education needs of low-income populations. However, this 
research spent little time investigating whether financial 
education resulted in positive behavior change for program 
participants. 

Other studies have focused more on behavior change by 
examining how financial education affects spending, 
savings, and debt management outcomes. Clancy, 
Gristein-Weiss, and Schreiner (2001) used data collected 
from the American Dream Demonstration and found that 
saving deposits and saving frequency in ID As increased as 
hours of financial education increased from 0 to 12 hours. 
However, after 12 hours, they found that the effect dimin¬ 
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ished and leveled off. A follow-up report by Schreiner, 
Clancy, and Sherraden (2002) showed that savings in¬ 
creased by only a small amount initially and then the effect 
leveled off after 8 to 10 hours of education. 

In a more recent study, Shockey and Seiling (2004) used 
the Transtheoretical Model of Change (TTM) to specifi¬ 
cally assess change in six financial behaviors over a 4- 
week period for individuals enrolled in an IDA financial 
education program. The TTM framework integrates major 
psychological theories into a theory of behavior change 
(i.e., Prochaska, 1979; Prochaska & DiClemente, 1983). 
The six financial behaviors examined were setting finan¬ 
cial goals, using a spending plan, tracking spending, reduc¬ 
ing debt, setting aside money for unplanned expenses, and 
saving money. Prior to the program, Shockey and Seiling 
found that program participants were, on average, at the 
stage of action with respect to reducing debt and at the 
stage of preparation for all other behaviors. Following the 
program, they found that participants experienced the 
smallest change for the behavior that was associated with 
reducing debt and the largest change for setting aside 
money for unplanned spending. 

Another study of low-income individuals, not tied to an 
IDA program, investigated the impact that financial educa¬ 
tion had on the decision of unbanked individuals to open a 
bank account and move into mainstream financial markets. 
Lyons and Scherpf (2004) showed that financial education 
was successful in encouraging the unbanked to open a 
bank account. However, they also found that, even after 
the program, financial constraints prevented a significant 
proportion of unbanked participants from opening an 
account. According to Lyons and Scherpf, no matter how 
much financial education some low-income individuals 
received, they were still unable to change some financial 
behaviors, because their overall financial position had not 
changed. They argued that the best measure of program 
“success” may be related to whether the participants re¬ 
ceive the financial skills needed to make decisions that are 
applicable to their specific financial circumstances. 

To summarize, the literature has provided evidence that 
financial education can improve the financial well-being of 
low-income individuals and their families. Yet, studies that 
have focused on low-income populations have been lim¬ 
ited in the following respects. First, most of these studies 
have relied on data collected solely from program partici¬ 
pants and not from any comparison groups. They com¬ 
pared only the pre- and post-program knowledge and 



behavior of program participants. However, in most cases, 
there was significant variation in the amount of financial 
education each participant received even within the same 
program, which offered an alternative to comparison group 
data. In addition, the administration of these programs was 
often varied, especially with respect to how participants 
were recruited and what portions of the curriculum were 
delivered by the instructors. Some programs offered “one- 
shot” workshops and seminars that focused on one or two 
lessons from the program, whereas others offered a more 
comprehensive program with multiple lessons. Only re¬ 
cently have researchers and financial professionals ques¬ 
tioned whether the amount of financial education matters 
(Clancy et al., 2001; Schreiner et al., 2002). In other 
words, is more better? A few studies have used hours of 
education to account for the amount of financial education 
received by program participants. However, there are other 
measures that could be used, such as the number of lessons 
completed. Using alternative measures allows us to com¬ 
plement the absence of control groups by comparing the 
participants who completed more lessons to those who 
completed less. It also adds to our understanding of 
whether more really is better. Determining the optimal 
amount of education can have important implications for 
program delivery. 

A second limitation of previous literature is that it has 
focused on the impact that financial education has on the 
knowledge and behaviors of only the program participants. 
Most financial education programs that have targeted low- 
income populations, however, have had a train-the-trainer 
component, where financial educators trained agency staff 
from non-profit organizations so that they, in turn, could 
effectively deliver the program to their clients. During 
these training programs, the agency staff actually com¬ 
pleted the program. They also learned about effective 
delivery methods and received guidance on how to re¬ 
spond to clients’ needs and how to motivate them to 
change their behaviors. Little, if any, research has exam¬ 
ined the impact that financial education has on the agency 
staff trained to deliver the programs (Baron-Donovan, 
Wiener, Gross, & Block-Lieb, 2005; Shelton & Moss, 
2002). These instructors play a critical role with respect to 
the quality of the program and whether program partici¬ 
pants are motivated to positively change their behaviors. 
Instructors who go through the program themselves are 
likely to be more confident in their own financial manage¬ 
ment skills and in their ability to respond to participants’ 
questions. The end result is that low-income audiences are 
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likely to have a more meaningful learning experience and 
to report improvement. 

This study builds upon prior research and addresses these 
critical gaps in the literature. Using a RPT, we controlled 
for prior financial behaviors and investigated the impact 
that the program had on both program participants and 
agency staff. We also used the total number of financial 
education lessons completed to provide additional insight 
into the relationship between the amount of financial 
education received and self-reported improvement in 
individuals’ actual and anticipated financial behaviors. 

Data Collection 

Description of the Program 

All My Money is a financial education program developed 
by University of Illinois Extension that focuses on provid¬ 
ing financial management and consumer skills to low- 
income households. The curriculum consists of eight 
instructor-led lessons that cover a number of personal 
finance topics including (a) making spending choices, (b) 
envelope budgeting, (c) planning expenditures, (d) under¬ 
standing credit, (e) handling credit problems, (f) building 
consumer skills, (g) taking consumer action, and (h) man¬ 
aging a checking account. Each lesson consists of hands- 
on activities and handouts as well as lesson plans and 
instruction guides. Also, each lesson is designed to take 
approximately 60 minutes to deliver. 

The program was primarily designed to target two audi¬ 
ences: (a) staff of social service organizations and govern¬ 
ment agencies that worked directly with low-income 
audiences and who were trained to deliver the program to 
their clients and (b) low-income clientele who may have 
had limited financial literacy. Extension educators trained 
agency staff members using a series of workshops that 
totaled 16-20 hours of hands-on instruction in basic finan¬ 
cial management depending on the number of lessons 
offered by the instructors. Roughly 95% of the agency 
staff were trained in the entire program and received all 
eight lessons. Trainings occurred over the span of a few 
days to a few weeks. 

The agency staff, in turn, offered the program to their 
clientele. Agency staff, however, had considerable discre¬ 
tion over how clients were recruited and which lessons 
were delivered to their clients. Therefore, the number and 
types of lessons offered to the clientele varied significantly 
by instructor and location. For example, the data showed 
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that more than half of the clients who participated in the 
program were taught the budgeting, planning, and credit 
lessons, whereas the banking and problem-handling les¬ 
sons were taught less frequently. Thus, the program lasted 
anywhere from a few days to a few weeks depending on 
how many lessons were administered to the clientele. 

To date, the program has primarily been offered in the 
state of Illinois. Over 100 agencies and organizations in 
Illinois have participated in the All My Money program 
including, but not limited to welfare-to-work and other 
social service programs, IDA programs, consumer credit 
counseling services, homebuyer education programs, 
community and faith-based organizations, and financial 
institutions. 

How the Evaluation Was Conducted 
Between 1998 and 2002, University of Illinois Extension 
educators collected data in Illinois from both the agency 
staff and clientele. The data were gathered using a RPT, 
where participants were asked at the end of the program to 
retrospectively report their financial behaviors prior to the 
program and how they would change as a result of the 
program. Given the recent movement towards more rigor¬ 
ous program evaluations that include control groups and a 
longitudinal component (Lyons, 2005; Lyons et al., 2006), 
the RPT may be perceived as less rigorous, and therefore 
less convincing, than other approaches. One concern in 
using the RPT is that program participants may be more 
inclined to show a learning effect, especially if the partici¬ 
pants develop a good rapport with their instructor. There 
are additional concerns that using a RPT could affect the 
validity of the data by introducing self-reporting and recall 
bias. 

However, given the realities of program evaluation for 
organizations that target low-income populations, the RPT 
is a common and useful evaluation tool. One clear advan¬ 
tage is that it only has to be administered once, reducing 
the amount of time the participants spend as research 
subjects. Another advantage is that it can reduce response- 
shift effects, which occur when a participant’s frame of 
reference changes significantly during the program 
(Howard, 1980; Lamb, 2005; Lamb & Tschillard, 2005). 
For example, participants may not be familiar with basic 
terms and concepts prior to the program. Therefore, a 
traditional pre-test may yield biased results if the test 
introduces terms and concepts before the participants are 
ready for them. The RPT reduces the likelihood of re¬ 
sponse-shift effects by clearing up misconceptions before 
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participants are asked to make assessments. Also, if the 
time lapse between the “before” and “after” is only a few 
days or a couple of weeks, the likelihood of recall bias will 
be lower, as was the case in the current study. 

The RPT included a self-assessment of how overall finan¬ 
cial management performance changed after the program. 
Information was also collected on anticipated changes in 
specific financial behaviors. The overall impact of the 
program was measured using the question, “After partici¬ 
pating in the program, how much would you say your 
ability to manage money has changed?” Participants re¬ 
ported their changes on a 5-point scale ranging from much 
worse to much better. The questionnaire also asked about 
respondents’ self-assessment with respect to the following 
five behavior categories: budgeting, intra-household com¬ 
munication, bill payment, ability to handle consumer 
problems, and comparison shopping. Participants indicated 
on a 4-point scale how often they engaged in each behav¬ 
ior prior to the program and how often they planned to 
engage in each behavior now that they have completed the 
program. Responses ranged from almost always to almost 
never. Additional information was obtained on each par¬ 
ticipant’s age, gender, family size and composition, per¬ 
sonal and household income, and educational attainment. 
The location and dates of the program, including number 
of lessons completed, were also recorded for each partici¬ 
pant. 

The process for collecting the survey information was as 
follows. Extension educators administered the RPT to 
agency personnel at the end of the train-the-trainer pro¬ 
gram. When the agents delivered the program to their 
clients, they had the option of administering the same RPT 
to those who completed the program. Agents who adminis¬ 
tered the evaluation to their clients were asked to return the 
surveys to the extension educator who had trained them. 

Of the more than 100 agencies who participated in the 
training, only 19 agencies chose to return surveys. Because 
it was optional, the agencies that returned the surveys may 
not have been random. In addition, their clientele may 
have differed in a fundamental way from the clientele from 
the agencies that did not return their surveys. We com¬ 
pared the characteristics of the agencies that returned 
surveys with those that did not and found that the agencies 
that had returned clientele surveys were fairly representa¬ 
tive of the agencies that had not. Key characteristics that 
were compared included size of the organization, target 
audiences, types of programs offered, and services pro- 
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vided to clientele. The composition of the clientele also 
tended to be fairly representative. 

In the end, a total of 763 evaluations were collected, 546 
from agency staff and 217 from clientele. Due to incom¬ 
plete and missing information on key survey questions, 

174 observations were unusable. The final sample con¬ 
sisted of 589 observations (77.2% of the 763 collected 
surveys), where 428 were agency personnel (72.7%) and 
161 were clientele (27.3%). 

Sample Description 

Table 1 presents demographic information on the agency 
personnel who participated in the training and the clientele 
who participated in the program. The first set of columns 
presents data for the pooled sample of agency personnel 
and clientele. The second set of columns is restricted to 
clientele participants only. One might question why the 
sample of agents and clientele were pooled. There were 
several reasons for this. First, recall that roughly 95% of 
the agents were trained in all eight lessons. This provided 
insufficient variation to separately measure the impact of 
the number of lessons on agency staff. In addition, the 
pooled regression findings showed that the effects of the 
program on participants’ behaviors did not vary signifi¬ 
cantly between the agency staff and the clientele. Addi¬ 
tional qualitative evidence suggested that the financial 
behaviors of the agents may not have been all that different 
from those of the clients. A number of the agents may have 
been struggling financially themselves, even though they 
may have been more financially knowledgeable and may 
have had more education and income than their clients. 
However, given that there was substantial variation in the 
number of lessons completed by the clientele, which is the 
ultimate target audience, we also include information in 
the descriptive tables on the clientele-only sample because 
this population may be of greater policy interest. Addi¬ 
tional explanation for why we pooled the data is provided 
later in the paper. 

On average, the pooled sample completed 6.7 lessons. 
Participants were comprised predominantly of female 
(86.9%) and middle-aged participants. The average age of 
the participants was 39 years, with 83.1% of the sample 
falling into the 25-54 age bracket. Only 8.2% of the entire 
sample did not complete high school, whereas 44.3% had 
received a college degree. With respect to family size and 
composition, 59.8% reported having at least one child 
living in the household, and 70.6% reported having 
spouses or other adult household members. Nearly half 
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(47.4%) of the pooled sample reported personal monthly 
earnings of $1,500 or more (the top income bracket in the 
survey), and 45.8% claimed household income from other 
sources besides their own earnings. Most (60.3%) of the 
trainings and programs were delivered in the Chicago area, 
the largest metropolitan area in the state. 

On average, clientele participants completed 3.7 lessons. 
The average age was 34.5 years, with the majority (58.4%) 
under 35 years of age. More than a quarter did not com¬ 
plete high school, whereas only 11.8% had a college 
degree. In addition, the majority (59.1%) of clientele 
participants reported monthly incomes below $1,000, and 
nearly three-quarters (73.9%) of them did not have any 
other source of household income. Approximately 88% 
were female, and 79.5% reported having children living in 
the household. Only 58.4% reported living with another 
adult. Thus, the client sample consisted of a large number 
of single mothers. With respect to location, 82% were 
located in the Chicago metropolitan and surrounding areas. 

Table 1 also provides demographic information for partici¬ 
pants who reported and did not report a positive improve¬ 
ment in their overall financial behaviors following the 
program. Approximately 47.0% of the pooled sample 
reported that as a result of the program their ability to 
manage their money was “much better,” 43.0% reported 
that it was “a little better,” 9.9% indicated that it was 
“about the same,” and only 0.1% indicated that it was “a 
little worse” or “much worse.” Participants who reported 
that their ability to manage their money was “a little bet¬ 
ter” or “much better” were classified as having experi¬ 
enced a positive improvement in their overall financial 
behavior. 

The majority of program participants reported an improve¬ 
ment in their financial management practices: 90.0% in the 
pooled sample (n = 530) and 85.0% in the clientele-only 
sample (n = 137). Note, however, that although clientele 
participants accounted for only 27.3% of the overall sam¬ 
ple, they made up 40.7% of those reporting no improve¬ 
ment. This perhaps suggests that either the program was 
not as successful for the clientele participants or they could 
not accurately assess their level of improvement. Due to 
the small number of participants reporting no improve¬ 
ment, however, one must be cautious in drawing infer¬ 
ences from these numbers. 

Two-tailed t tests were used to identify characteristics that 
were significantly different between those who reported an 
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Table 1. Description of the Sample by Overall Improvement in Financial Behavior 


Pooled sample Clientele-only sample 


Financial behavior Financial behavior 

after the program after the program 

Variables Total Total 



ON 

OO 

IT) 

II 

No 

improve 
(« = 59) 

Improve 
(« = 530) 

1*1 

(V=161) 

No 

improve 
(i n = 24) 

Improve 
(n= 137) 

\t\ 

Total number of lessons 

6.7 

5.6 

6.8 

4.08** 

3.7 

2.9 

3.9 

2.16** 

Age 

39.0 

39.1 

39.0 

0.06 

34.5 

34.1 

34.6 

0.17 

24 or less 

8.7 

11.9 

8.3 

0.92 

20.5 

25.0 

19.7 

0.59 

25-34 

31.1 

23.7 

31.9 

1.28 

37.9 

33.3 

38.7 

0.50 

35-44 

28.2 

32.2 

27.7 

0.72 

23.6 

25.0 

23.4 

0.17 

45-54 

23.9 

23.7 

24.0 

0.04 

13.7 

8.3 

14.6 

0.82 

55 or more 

8.2 

8.5 

8.1 

0.10 

4.3 

8.3 

3.6 

1.03 

Female 

86.9 

91.5 

86.4 

1.10 

88.2 

87.5 

88.3 

0.11 

Education 

Less than high school 

8.2 

10.2 

7.9 

0.60 

26.1 

25.0 

26.3 

0.13 

High school or GED 

20.5 

28.8 

19.6 

1.66* 

35.4 

37.5 

35.0 

0.23 

Some college 

27.0 

18.6 

27.9 

1.52 

26.7 

25.0 

27.0 

0.20 

Bachelor’s degree 

27.7 

23.7 

28.1 

0.71 

8.7 

8.3 

OO 

OO 

0.07 

Graduate degree 

16.6 

18.6 

16.4 

0.44 

3.1 

4.2 

2.9 

0.32 

Number of children 

None 

40.2 

45.8 

39.6 

0.91 

20.5 

29.2 

19.0 

1.14 

1 

21.7 

13.6 

22.6 

1.61 

18.6 

12.5 

19.7 

0.83 

2 

21.4 

22.0 

21.3 

0.13 

31.1 

33.3 

30.7 

0.26 

3 or more 

16.6 

18.6 

16.4 

0.44 

29.8 

25.0 

30.7 

0.56 

Other adult members 

70.6 

83.1 

69.2 

2.21** 

58.4 

83.3 

54.0 

2.73** 

Personal income 

$249 or less 

3.2 

1.7 

3.4 

0.70 

9.9 

4.2 

10.9 

1.02 

$250-499 

5.8 

13.6 

4.9 

2.72** 

19.3 

25.0 

18.2 

0.77 

$500-749 

7.8 

10.2 

7.5 

0.71 

19.3 

25.0 

18.2 

0.77 

$750-999 

8.0 

6.8 

8.1 

0.36 

10.6 

16.7 

9.5 

1.05 

$1,000-1,249 

12.6 

15.3 

12.3 

0.66 

10.6 

12.5 

10.2 

0.33 

$1,250-1,499 

15.3 

11.9 

15.7 

0.77 

11.2 

4.2 

12.4 

1.18 

$1,500 or more 

47.4 

40.7 

48.1 

1.08 

19.3 

12.5 

20.4 

0.91 

Other income sources 

45.8 

57.6 

44.5 

1.92* 

26.1 

41.7 

23.4 

1.89* 

Chicago area 

60.3 

62.7 

60.0 

0.40 

82.0 

79.2 

82.5 

0.39 

Clientele 

27.3 

40.7 

25.8 

2.43** 

100.0 





Note. Mean values are reported for the variables that represent total number of lessons and age. 
*p < .10. **p < .05. 
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improvement and those who did not. Note that participants 
who reported an improvement completed more lessons on 
average than those who showed no improvement. In the 
pooled sample, those who reported improvement attended 
6.8 lessons, compared to 5.6 for those who reported no 
improvement. For the clientele, the average number of 
lessons completed was 3.9 and 2.9, respectively. The 
differences between the two groups were significant at 
the .05 level. 

Program participants who reported an improvement were 
also less likely than those who reported no improvement to 
live in a household with other adult members including a 


spouse or parent (p < .05). In addition, they were less 
likely to report having other income sources besides per¬ 
sonal income (p < .10). Further evidence from the pooled 
sample revealed that those who reported an improvement 
were less likely to be a client than those who reported no 
improvement (p < .05). 

Table 2 presents evidence of program impact according to 
anticipated changes in specific financial behaviors. Over¬ 
all, the program appears to have had a positive impact on 
each of the five behaviors. The most noticeable improve¬ 
ment was for participants who reported that they antici¬ 
pated running out of money less frequently after the pro- 


Table 2. Anticipated Changes in Specific Financial Behaviors 


Financial behaviors “ 

Pooled sample (%) 

Clientele-only sample (%) 


pre 

post 

pre 

post 

Do not run out of money (n = 549) 

1 = Almost never 

10.9 

1.5 

26.8 

1.4 

2 = Sometimes 

16.2 

3.5 

26.8 

6.3 

3 = Often 

39.3 

30.6 

32.4 

48.6 

4 = Almost always 

33.5 

64.5 

14.1 

43.7 

Talk with family about money (7? = 555) 

1 = Almost never 

19.1 

7.2 

28.7 

12.6 

2 = Sometimes 

45.2 

24.7 

38.5 

30.8 

3 = Often 

22.5 

40.4 

18.2 

28.0 

4 = Almost always 

13.2 

27.8 

14.7 

28.7 

Do not pay bills late (n = 551) 

1 = Almost never 

8.0 

1.8 

20.0 

3.6 

2 = Sometimes 

9.6 

3.3 

16.4 

6.4 

3 = Often 

35.4 

17.1 

34.3 

29.3 

4 = Almost always 

47.0 

77.9 

29.3 

60.7 

Complain when having a consumer problem (n = 553) 

1 = Almost never 

25.5 

13.9 

24.1 

24.1 

2 = Sometimes 

44.7 

28.0 

41.8 

39.7 

3 = Often 

16.1 

28.9 

20.6 

20.6 

4 = Almost always 

13.7 

29.1 

13.5 

15.6 

Compare prices and quality before buying (n = 556) 

1 = Almost never 

7.9 

2.2 

12.7 

4.9 

2 = Sometimes 

29.5 

8.3 

29.6 

10.6 

3 = Often 

25.4 

23.7 

20.4 

23.9 

4 = Almost always 

37.2 

65.8 

37.3 

60.6 


Note. The number of observations varies slightly for each behavior because a few participants chose not to respond to all 
financial behavior questions. 
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gram, and this improvement was more pronounced for the 
clientele. Over half of the clientele reported not running 
out of money “almost never” or “sometimes” prior to the 
program, compared to only 7.8%, who anticipated not 
running out of money following the program. 

Participants, especially clientele participants, reported 
similar improvement in paying bills on time. For the 
clientele, 29.3% reported that they “almost always” did not 
pay bills late prior to the program, compared to 60.7% who 
anticipated “almost always” paying bills on time after the 
program. Participants also experienced improvements in 
two other behavior categories—financial communication 
within the family and comparison shopping. 

Table 3 examines the relationship between the total num¬ 
ber of lessons completed (or the amount of financial edu¬ 
cation received) and the self-reported improvement for 
each of the five financial behaviors. Information is also 
included on pre-program behavior, where pre-program 
behavior is defined to be the self-reported degree to which 
the participant engaged in a particular financial behavior 
prior to the program. For each behavior, the sample ex¬ 
cludes those who responded that they “almost always” 
engaged in that particular financial behavior prior to the 
program. For the pooled sample, improvement in each of 
the behaviors was associated with a higher average number 
of lessons completed, with the exception of the financial 
behavior “do not run out of money.” A similar, but gener¬ 
ally weaker trend was found for three of the five financial 
behaviors for the clientele sample. For the clientele-only 
sample, improvement in talking with the family about 
money, handling consumer problems, and comparison 
shopping were associated with more lessons on average. 

Table 3 also shows that the greatest improvement occurred 
among participants who reported the poorest financial 
behaviors prior to the program. In both samples, over 
80.0% (88.5% in the pooled sample and 83.9% in the 
clientele sample) of those reporting no improvement in the 
“do not run out of money” category indicated that they 
“often” did not run out of money prior to the program. In 
addition, it should not be surprising that, for the clientele 
sample, 78.1% of those who reported an improvement in 
the “do not run out of money” category indicated that their 
pre-program behavior in this area was poor (i.e., they 
“almost never” or “sometimes” did not run out of money). 
It should be noted, however, that some of these descriptive 
differences disappeared when we controlled for other 
factors in the multivariate analysis. 
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Empirical Framework 

Although we did not have access to a traditional control 
group (i.e., a sample of non-participants), we were able to 
assess the treatment effect of the program by looking at 
whether the program impact was larger for participants 
who completed a greater number of lessons. The relation¬ 
ship was expressed as follows: 

Yj* = a ■ Lessonsi + X ; p + T5 + e,-, (1) 

where 7, = 1 iff Y t >0 and 0 otherwise. 

The subscript /' indexed individual participants, for i = 1, 

2, .... N. In this model, Yj* was the improvement in finan¬ 
cial management behaviors, that indicated the degree to 
which participants anticipated that they would engage in 
more desirable financial behaviors as a result of the pro¬ 
gram. Yj* was a latent measure that was not directly ob¬ 
servable. Instead, a binary index, Yj, was observed in the 
data such that Yj was equal to one if the i th participant 
reported any positive change in his/her financial behavior 
following the program and zero otherwise. 

Yj* was modeled as a function of the total number of 
lessons completed by each program participant (Lessons), 
a vector of demographic and economic characteristics of 
the participant (X), and a vector of fiscal year dummies 
(T). Included in X were control variables such as age, 
gender, education, household size and composition, per¬ 
sonal income of the participant, and whether the household 
had other income sources. We also included in the model 
an indicator for whether the participant resided in the 
Chicago metropolitan or surrounding areas, because we 
suspected that the overall economic and financial environ¬ 
ment that households faced in large metropolitan areas was 
considerably different from that found in more rural areas. 
One might question why the model did not control for 
race/ethnicity, marital status, employment patterns, and 
some measure of asset holdings or net worth. Unfortu¬ 
nately, the data set did not include this information. Our 
task was to gain as much information as possible from the 
available data that had been collected. 

To determine if the base level of financial knowledge and 
experience varied between the clientele and agency staff, 
we also included an indicator variable that identified 
whether the participant was a client or agent. The signifi¬ 
cance of this dummy variable determined whether there 
were differences in the parameters (a, P, and 5) between 
those who showed improvement and those who did not, 
which in turn indicated whether separate models should be 



Table 3. Anticipated Changes in Specific Financial Behaviors by Total Number of Lessons and 
Pre-Program Financial Behaviors, Reduced Sample 



Do not run out 
of money 

Talk with family 
about money 

Do not pay 
bills late 

Complain when 
problem 

Compare prices 
and quality 


No 

improve 

Improve 

No 

improve 

Improve 

No 

improve 

Improve 

No 

improve 

Improve 

No 

improve 

Improve 

Pooled sample 

n 

104 

261 

221 

261 

70 

222 

216 

261 

90 

259 

Total number of 
lessons completed 

6.6 

6.5 

6.7 

7.1* 

6.1 

6 . 6 * 

6.3 

7 4 ** 

6.6 

7.1* 

Pre-program 
behavior (%) 











1 = Almost never 

2.9 

21 . 8 ** 

13.6 

29.1** 

4.3 

18.5** 

19.9 

37.5** 

3.3 

15.8** 

2 = Sometimes 

8.7 

30.7** 

46.2 

57.1* 

10.0 

20.7* 

52.8 

51.0 

40.0 

49.4 

3 = Often 

88.5 

47.5** 

40.3 

13.8** 

85.7 

60.8** 

27.3 

11.5** 

56.7 

34.8** 

Clientele-only sample 

n 

31 

91 

63 

59 

29 

70 

84 

38 

26 

63 

Total number of 
lessons completed 

3.8 

3.7 

3.7 

3.8 

4.1 

3.6 

3.8 

4.1 

3.6 

4.2 

Pre-program 
behavior (%) 











1 = Almost never 

6.5 

39.6** 

20.6 

47.5** 

10.3 

35.7* 

22.6 

39.5* 

7.7 

25.4* 

2 = Sometimes 

9.7 

38.5** 

46.0 

44.1 

13.8 

27.1 

47.6 

50.0 

38.5 

50.8 

3 = Often 

83.9 

22 . 0 ** 

33.3 

8.5** 

75.9 

37.1** 

29.8 

10.5** 

53.8 

23.8** 


Note. The sample was reduced to those who did not respond “almost always” for their pre-program behavior, and thus the 
category “almost always” is excluded from the table. The breakdown of other sample characteristics by behavior and level 
of improvement is available upon request. 

*p < .10. **p < .05. 


estimated for the agency staff and the clientele. The vector 
T controlled for economic conditions specific to each 
survey year, as well as for yearly variation in audience 
makeup, program budgets, resources available for training, 
and the ease of participants’ behavioral adjustments. 

For each model, the unknown parameters (a, P, and 5) 
were obtained using the probit method. The error terms, e\, 
were assumed to be random and normally distributed with 
a mean of zero. The coefficient of interest, a, was expected 
to be positive and significant, which meant that the more 
lessons participants completed, the more likely they were 
to engage in more desirable financial behaviors. Two 
probit models were estimated to determine the effect that 
financial education had on (a) overall financial behavior 
and (b) anticipated change in specific financial behaviors. 
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The remainder of this section presents the specifics for the 
two models. 

Model 1: Overall Program Impact 
Recall that the overall program effect was measured by 
the question, “After participating in the program, how 
much would you say your ability to manage money has 
changed?” Out of the five ordered categories ranging from 
much worse to much better , the responses “a little better” 
and “much better” were considered to demonstrate positive 
latent effects. The probability that the program had a 
positive effect overall was modeled as follows: 

Pr(Yj= 1 | Lessons i, X ; , T) = <P(ccLessonSj + X ; p + T5), (2) 

where @(.) was the standard normal distribution func¬ 
tion. 
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Note that this model did not control for an individual’s 
prior financial knowledge and habits. Although demo¬ 
graphic and socio-economic control variables were in¬ 
cluded in the model to account for sample heterogeneity, 
prior differences in overall financial education across 
participants were not observed. It was important, however, 
to assess the program effect holding constant the partici¬ 
pants’ financial habits and practices prior to the program. 
We addressed this issue by estimating a second model that 
looked at anticipated changes in specific financial behav¬ 
iors. 

In addition to running the probit models, we estimated a 
series of ordered probit models. Using the participants’ 
responses to the 5-point scale, we created a three-category 
dependent variable that was equal to 1 if “much better,” 2 
if “a little better,” and 3 if “about the same or worse.” The 
coefficients obtained from the ordered probit were similar 
to those for the probit. The probit, however, was found to 
be a better fit for the distribution of the data. The findings 
from the probit models are presented in the results section; 
the findings from the ordered probits are available from 
the authors upon request. 

Model 2: Changes in Specific Financial Behaviors 
As previously mentioned, participants were asked to 
evaluate their financial practices in five behavioral cate¬ 
gories using a 4-point scale to indicate how often they 
engaged in each behavior prior to the program and how 
often they planned to engage in each behavior following 
the program. Responses ranged from almost always to 
almost never. Using this information, we constructed a 
binary dependant variable to indicate improvement for 
each behavioral category. Specifically, Y t equaled one if 
posti> prei and 0 otherwise. Prei and posti were the self- 
reported levels of financial practice of the i' h participant 
before the program and the anticipated level of financial 
practice after the program, with the higher value denoting 
more desirable financial behaviors. 2 Because the partici¬ 
pant’s prior skill levels could affect the program’s impact, 
dummy variables that controlled for financial behaviors 
prior to the program were included in the model such 
that 

Pr(Yi= 1 | Lessonsi, X;, T ,prel) = (3) 

<L>(a ■ Lessonsi + £ 9jl [prei =j] + X$ + T5), 

j 

where ( I >(.) was the standard normal distribution 
function. 
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In this model, j was equal to 1 for “poor improvement,” 2 
for “moderately poor improvement,” 3 for “moderately 
good improvement,” and 4 for “good improvement.” ![.] 
was an index function that took the value of one if its 
argument was true and zero otherwise. The parameters 0j 
for j = 1, 2, 3 represented how the likelihood of improve¬ 
ment depended on the participant’s level of financial 
behavior prior to the program. The parameters were ex¬ 
pected to decrease in j, which implied that the program 
was more likely to benefit those who started out at lower 
levels of financial knowledge (see the Appendix for 
mathematical proof). Because no improvement was possi¬ 
ble for those who reported the highest level of financial 
knowledge prior to the program {prei = 4), Equation 3 was 
estimated for the sample restricted to those with prei < 4. 

A significant and positive a would suggest that, for a given 
level of prior financial skill, an additional lesson would 
increase the probability that participants anticipated an 
improvement in their financial behaviors upon completing 
the program. 

Results 

Tables 4 and 5 present the estimation results for (a) the 
likelihood that the program had a positive impact on over¬ 
all behavior and (b) the likelihood that the program re¬ 
sulted in improvement in specific financial behaviors. 
Coefficients, standard errors, and marginal effects are 
presented in both tables. Marginal effects were calculated 
at the sample means. Recall that we were unable to run 
separate regressions for the agents and clientele due to data 
constraints related to the limited variation in the number of 
lessons completed by the agents. For this reason, models 
were estimated for (a) the pooled sample of agency staff 
and clientele and (b) the clientele-only sample. Note that 
for the pooled regression in Table 4 the dummy variable 
for clientele was insignificant. This finding suggested that 
the impact of the program on the behaviors of the clientele 
was not significantly different from that of the agents, 
providing empirical support for the pooled model. Also, 
recall that for Table 5 the sample was reduced to the par¬ 
ticipants who had less-than-perfect financial management 
skills prior to the program (prei < 4). 

Overall Program Impact 

Table 4 shows that, although the coefficient on the total 
number of lessons completed was positive for both the 
pooled and clientele-only samples, it was only significant 
at conventional levels for the pooled sample. The marginal 
effect of an additional lesson at the mean for the pooled 
sample was 1.5 percentage points. This suggested that, in 
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Table 4. Probit Regressions for the Positive Overall Program Effect 


Variables 


Pooled sample 
(n = 589) 


Clientele-only sample 
(« = 161) 


Coeff 

(SL) 

dY/dX 

Coeff 

(SL) 

dY/dX 

Total number of lessons 

0.135 

(0.053)** 

0.015 

0.191 

(0.188) 

0.011 

Age 







25-34 

0.430 

(0.315) 

0.044 

0.786 

(0.566) 

0.039 

35-44 

-0.048 

(0.310) 

-0.006 

0.182 

(0.554) 

0.009 

45-54 

0.232 

(0.332) 

0.024 

0.932 

(0.771) 

0.029 

55 or more 

0.199 

(0.413) 

0.020 

-0.062 

(0.846) 

-0.004 

Female 

-0.486 

(0.291)* 

-0.042 

-0.913 

(0.687) 

-0.027 

Education 







Less than high school 

0.100 

(0.326) 

0.011 

-0.002 

(0.469) 

- 0.000 

Some college 

0.513 

(0.251)** 

0.049 

0.656 

(0.549) 

0.028 

Bachelor’s degree 

0.160 

(0.250) 

0.017 

0.239 

(0.891) 

0.011 

Graduate degree 

-0.208 

(0.281) 

-0.026 

-0.354 

(1.025) 

-0.027 

Number of children 







1 

0.456 

(0.251)* 

0.043 

0.960 

(0.643) 

0.032 

2 

0.250 

(0.235) 

0.026 

1.132 

(0.578)* 

0.048 

3 or more 

0.044 

(0.253) 

0.005 

0.916 

(0.563) 

0.039 

Other adult members 

-0.355 

(0.228) 

-0.036 

-0.553 

(0.490) 

-0.029 

Personal monthly income 







$500-749 

0.159 

(0.363) 

0.016 

-0.174 

(0.510) 

-0.011 

$750-999 

0.631 

(0.405) 

0.048 

0.059 

(0.650) 

0.003 

$1,000-1,249 

0.120 

(0.355) 

0.013 

0.110 

(0.714) 

0.006 

$1,250-1,499 

0.707 

(0.371)* 

0.056 

2.354 

(0.964)** 

0.042 

$1,500 or more 

0.306 

(0.331) 

0.035 

1.097 

(0.693) 

0.036 

Other income sources 

-0.405 

(0.193)** 

-0.048 

-0.851 

(0.449)* 

-0.073 

Chicago area 

-0.419 

(0.206)** 

-0.045 

-1.462 

(0.718)** 

-0.042 

Clientele 

0.176 

(0.341) 

0.019 




Year dummies (1998-2000) 

Yes 



Yes 



Constant 

1.228 

(0.686)* 


2.980 

(1.320)** 


Predicted probability at X-bar 


0.943 



0.977 


Pseudo R-squared 

.182 



.434 



Log likelihood 

-156.800 



-38.357 




Note. We also estimated ordered probits using the dependent variable in three categorical responses, and the coefficients 
were very similar to the findings for the probits. Base categories include age (24 or less), high school or GED, no child¬ 
ren, and personal income ($499 or less). In the clientele-only regression, a dummy variable for 1998 is dropped from the 
regression to avoid perfect collinearity. Robust standard errors are reported, and marginal effects are calculated at the mean 
values. 

*p < .10. **p < .05. 
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terms of overall program impact, an additional lesson may 
have resulted in a more positive improvement in overall 
financial behavior for agency personnel than for clientele. 
However, the magnitude of the effect of the lessons for the 
pooled sample was not large. An additional lesson at the 
sample mean (6.7 lessons) increased the likelihood that 
participants experienced an improvement in their overall 
financial behavior by 1.6%, off a baseline probability of 
94.3%. 

We considered the possibility that the effect of the number 
of lessons may not have been linear and explored alterna¬ 
tive specifications by including a quadratic term for the 
number of lessons. The probit coefficient for the quadratic 
term was insignificant. To determine if the specific lessons 
that the participant completed had an impact on his/her 
behaviors, we also estimated the model using dummy 
variables for individual lessons instead of a single variable 
for the total number of lessons. The results showed that the 
quantity of lessons received was more important than the 
specific type of lesson. In fact, there was little evidence 
that specific types of lesson resulted in improved financial 
behaviors. 

With respect to factors other than the number of lessons, 
we found that for the pooled sample females were signifi¬ 
cantly less likely than males to report an improvement in 
their overall financial behavior (p < .10). Given the small 
number of male participants in the sample, however, this 
finding may not have been representative of the low- 
income male population as a whole. With respect to educa¬ 
tion, only the category “some college” was positive and 
significant (p < .05), suggesting the program effect was 
greatest for those with some college education. 

Participants with children were somewhat more likely than 
participants without children to experience an overall 
improvement (p < .10), which implied that financial educa¬ 
tion program for adults may indirectly help children in 
poor families. Compared to the lowest income category 
(less than $499 per month), higher levels of income in¬ 
creased the probability of a positive program impact. 
However, the income bracket ($1,250-1,499 per month) 
had a significant coefficient (p < .10 for the pooled sample 
and p < .05 for the clientele-only sample). For both the 
pooled and clientele-only samples, access to other house¬ 
hold income sources as well as residing in the Chicago 
metropolitan or surrounding areas significantly decreased 
the likelihood that participants reported overall improve¬ 
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ment in their financial behaviors, suggesting difficulties in 
financial education for the working poor in urban settings. 

Changes in Specific Financial Behaviors 
Table 5 presents the regression results for the five financial 
behaviors. The coefficients and marginal effects are pre¬ 
sented for the number of lessons (a) and the indicator 
variables that control for levels of pre-program behavior 
(0i and 0 2 ). (9jwas the omitted base category and is there¬ 
fore not included in the table. The findings showed that the 
effect of the number of lessons was insignificant for all of 
the behaviors except comparison shopping. Controlling for 
demographic and economic characteristics, as well as for 
the level of comparison shopping before the program, 
additional lessons significantly increased the probability 
that a participant reported an improvement in comparison 
shopping by 4.7 percentage points. Thus, the number of 
lessons did not appear to play a significant role in improv¬ 
ing specific financial management behaviors. We suspect 
this was because the number of lessons completed may 
have been determined endogenously with the unobserved 
traits underlying participants’ ability to improve behaviors. 
That is, the number of lessons may have been negatively 
correlated with self-perceived pre-program behaviors. 

Also, once prior level was held constant, those who were 
the least likely to be able to improve their financial behav¬ 
iors may have been more likely to be taught more lessons. 
If this was the case, the program impact (as measured by 
the number of lessons completed) may have been absorbed 
in 8 and, therefore, may not have appeared in a. 

The significance of the coefficients for pre-program be¬ 
havior levels ( 6 1 and 0 2 ) showed that the probability of 
improvement was larger for participants who started out 
with poorer financial practices. This result was consistent 
with existing findings from evaluations of employer- 
provided financial education programs (Bayer et al., 1996; 
Bernheim & Garrett, 2003). For instance, participants who 
almost always ran out of money before the program were 
44.1 percentage points more likely to improve their budg¬ 
eting after they attended the lessons than those who had 
similar demographic and economic characteristics but 
seldom ran out of money. Similar results were found for 
communicating with family members about money, paying 
bills on time, filing consumer complaints, and comparison 
shopping, which suggests that participants who lacked 
financial skills prior to the program may have been more 
likely to gain from the program. As with the pooled sam¬ 
ple, the coefficients for low pre-program status ( 6 t and 0 2 ) 
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Table 5. Improvements in Specific Financial Behaviors, Reduced Sample 




Pooled sample 


Clientele-only sample 

Behaviors — 

Coeff 

(SE) 

dF/dX 

Coeff 

(SE) 

dF/dX 

Do not run out of money 







a 

-0.014 

(0.062) 

-0.004 

-0.081 

(0.187) 

-0.010 

0 i 

1.483 

(0.340)*** 

0.441 

2.462 

(0.794)*** 

0.288 

02 

1.201 

(0.222)*** 

0.357 

2.434 

(0.714)*** 

0.285 

n 

365 



122 



InL 

-171.929 



-28.742 



Talk with family about money 







a 

0.010 

(0.054) 

0.004 

-0.172 

(0.122) 

-0.069 

0 i 

1.430 

(0.199)*** 

0.566 

1.632 

(0.445)*** 

0.650 

02 

0.861 

(0.153)*** 

0.341 

0.869 

(0.395)** 

0.346 

n 

482 



122 



InL 

-283.685 



-63.074 



Do not pay bills late 







a 

0.028 

(0.070) 

0.008 

-0.003 

(0.147) 

-0.001 

0 i 

1.296 

(0.363)*** 

0.358 

0.649 

(0.497) 

0.183 

02 

0.880 

(0.276)*** 

0.243 

0.745 

(0.494) 

0.210 

n 

292 



99 a 



InL 

-133.379 



-38.831 



Complain when having 







consumer problems 







a 

0.094 

(0.058) 

0.037 

-0.001 

(0.109) 

- 0.000 

0 i 

1.095 

(0.197)*** 

0.434 

0.942 

(0.444)** 

0.311 

02 

0.551 

(0.174)*** 

0.218 

0.453 

(0.390) 

0.149 

n 

477 



122 



InL 

-278.658 



-58.478 



Compare prices and quality 







before buying 







a 

0.160 

(0.077)** 

0.047 

0.171 

(0.150) 

0.044 

0 i 

1.451 

(0.361)*** 

0.426 

1.663 

(0.589)*** 

0.423 

02 

0.399 

(0.169)** 

0.117 

0.811 

(0.448)* 

0.207 

n 

349 



89 b 



InL 

-170.208 



-34.721 




Note. The sample was reduced to those who did not respond “almost always” for their pre-program behavior. The omitted 
category for 0’s was pre ,• = 3 (0 3 ). Coefficients were obtained by controlling for age, gender, education, number of children, 
presence of other adult members, personal income, Chicago area indicator, clientele indicator, and the year fixed effects. 
Estimated coefficients and marginal effects for the control variables are available upon request. 

a Two control variables (personal income $1,000-$ 1,249 and the year dummy for 1998) were dropped from the regression to 
avoid perfect collinearity. b Two control variables (age 55 or more and the year dummy for 1998) were dropped from the 
regression to avoid perfect collinearity. 

*p < .10. **p < .05. ***p < .01. 
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were positive for the clientele-only sample, but the results 
tended to be somewhat less significant. 

Discussion 

The results of this study have useful implications for 
financial professionals and educators who are evaluating 
financial education programs that target low-income 
populations. The results of this study provide some evi¬ 
dence that the amount of financial education received (i.e., 
number of lessons) may result in an overall improvement 
in financial behavior. However, the findings seem to 
indicate that the prior level of financial experience may 
matter more than the number of lessons completed. Sig¬ 
nificantly greater impacts on behavior were observed for 
participants with lower levels of financial behaviors prior 
to the program, suggesting that the program was effective 
in reaching participants who were most in need of finan¬ 
cial education, including agency staff who reported poor 
pre-program behaviors. Note, however, that the same self- 
reported characterization of pre-program behavior by 
agency staff and clientele (i.e., the same pre0 may have 
represented different levels of actual behavior, especially 
if agents had higher levels of pre-program skills and 
knowledge than clients. 

The results of this study also suggest that the program had 
the largest impact on those financial behaviors that could 
most readily be altered in the short run. For instance, the 
impact of the number of lessons was significant for com¬ 
parison shopping—a behavior participants could immedi¬ 
ately improve regardless of their current financial circum¬ 
stances. Moreover, the largest marginal effects for pre¬ 
program behavior of the clients were found for those 
behaviors that could most readily be changed after the 
program (i.e., comparison shopping and talking with 
family about money), instead of those behaviors that were 
dependent on the participant’s financial situation (running 
out of money and paying bills on time) or personal circum¬ 
stances (dealing with consumer problems). 

Interestingly, neither the number of lessons nor the level of 
pre-program behavior had a significant impact on clientele 
participants paying their bills on time. However, for the 
pooled sample, poor pre-program behavior did result in a 
significant increase in paying bills on time. A plausible 
explanation is that, on average, the clientele may have 
been worse off financially than the agency staff because 
their actual financial position probably did not change as 
an immediate result of the program. Thus, even though 
clientele may have learned of the costs associated with late 
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payments from the program, they were less likely than 
agency staff to be in a financial position that would allow 
them to alter their ability to pay their bills on time. In other 
words, agency staff that missed bill payments may have 
just been lacking financial management skills, whereas 
clients who missed payments may have been more likely 
to be financially destitute. 

It is also interesting to note that running out of money is 
a behavior for which the pre-program level was highly 
significant for both the pooled and clientele-only samples. 
However, where one might expect to find a larger marginal 
effect, the marginal effect was lower for the clientele-only 
sample because they were more likely to start with lower 
levels of financial knowledge and thus had more room to 
improve. However, like paying bills on time, an individ¬ 
ual’s ability to not run out of money is also related to the 
financial situation of the participant. In this case, the 
agency staff were likely to be better off financially than the 
clientele, and thus in a better position to change behaviors 
related to their financial holdings. 

Overall, these findings suggest that financial education 
programs may want to distinguish between behaviors that 
can more easily be changed in the short run and behaviors 
that require more fundamental changes in other aspects of 
participants’ lives before they can be realized. Financial 
education programs that focus solely on behavioral goals 
that participants have little chance of implementing in the 
short term may run the risk of becoming irrelevant to their 
target audience. Participants may view the goals of the 
program as unattainable; some may even become discour¬ 
aged and not take any action to change their behaviors. 

Given our findings, financial professionals and educators 
need to more carefully assess how knowledge translates 
into behavior change for low-income populations and the 
instructors who deliver the programs. It is critical that 
researchers select outcomes and indicators that are appro¬ 
priate to the financial capabilities of their target audiences. 
It may be that some individuals, because of their particular 
financial situation, are unable to change certain financial 
behaviors no matter how much financial education they 
receive. Thus, if financial education does not result in 
behavior change, it may not be that the program is ineffec¬ 
tive. It may be that inappropriate outcomes and indicators 
were selected. Researchers may want to focus less on 
outcomes tied to individuals’ financial situations and more 
on whether individuals are able to make sound financial 
decisions regardless of their financial situation. They may 
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also want to use a wider set of program outcomes to ensure 
that the positive effects of these programs are not underes¬ 
timated. 

In addition, financial professionals and educators may 
want to re-examine the link between financial knowledge 
and behavior change, especially for program instructors. 
One might assume that instructors delivering programs to 
low-income audiences are knowledgeable and experienced 
financial educators. However, informal interviews con¬ 
ducted with extension educators, who trained the agents 
for this particular program, seem to reveal that this may 
not exactly be the case. It was reported that levels of 
financial knowledge and experience varied significantly 
for the agents—from having no financial education experi¬ 
ence to several years of experience. Extension educators 
reported that for some agents the training sessions served 
as a “refresher course” or as a “reinforcement” of basic 
financial concepts. However, others were seeing some of 
the information for the first time. One extension educator 
commented: 

We thought they knew this stuff when they came in, but 
we got agents who wrote in the comments that they 
felt they learned something from the program. They 
said they could use this information for themselves as 
much as for their clients. 

At the end of the program, agents were asked to comment 
on what they learned from the program and how they 
planned to use the information and materials they received. 
In general, several reported that they felt “more familiar” 
and “more comfortable” with the information, especially 
related to budgeting, savings, and credit management. One 
agent commented: 

I learned new avenues to save money personally and 
also new techniques and strategies to help my clients 
budget and learn new skills to save money and start 
savings or checking accounts. 

Another stated: 

I learned principles of effective money management 
that will help me to become more organized in devel¬ 
oping a system that I am comfortable with (a) in 
tracking my expenditures and (b) investing wisely, 
consistently, and enthusiastically. The info on credit 
card smarts. I will use this with clients as well as in 
my personal life. 


Extension educators also noted that there were agents who 
were experiencing financial problems and having difficulty 
managing their own finances. One extension educator 
commented: 

A number of the agency staff who deal with low- 
income clients are low paid and money is tight for 
them, and many are struggling financially themselves. 

Thus, the agents and clientele in our sample were more 
similar than one might initially think. In light of these 
informal interviews, our finding of no significant differ¬ 
ence in overall financial behavior between agents and 
clients for the pooled model may not be that surprising 
(i.e., recall that in Table 4 the dummy variable for clientele 
was found to be statistically insignificant). Although the 
agents may have been more financially knowledgeable 
than the clients on average, a significant number of the 
agents may have been struggling financially, even though 
they had more education and income than their clients. 
Thus, one must consider the possibility that providing 
more educational knowledge (“number of lessons”) may 
not necessarily be the appropriate catalyst to motivate 
individuals to change their financial behaviors. It may be 
that the level of financial experience matters more, as 
participants with the poorest pre-program behaviors gener¬ 
ally reported the greatest improvement. 

As with most program evaluations, one needs to be cau¬ 
tious in regarding these findings as conclusive. First, our 
clientele-only sample is somewhat small for making infer¬ 
ences about the program’s impact on the financial behav¬ 
iors of the low-income population as a whole. Recall, 
however, that the clientele sample is fairly representative 
of the clientele as a whole for all the agencies that partici¬ 
pated in a train-the-trainer program. Second, the treatment 
effect in our model is identified not by variation in the 
outcomes of those who underwent the treatment versus 
those who did not (i.e., participants versus non-parti¬ 
cipants), but rather by the variation in the outcomes of 
participants who underwent varying treatment intensities 
(measured by the number of lessons) and who started out 
with different pre-program behaviors. As a result, the 
effects we found may not reflect true treatment effects but 
rather marginal effects conditional on some level of ex¬ 
perience and participation in the program. 

Ideally, we would have liked to have had a control group 
and a larger sample size so we would not have had to pool 
the data. It would have been of particular interest to exam- 


42 


Financial Counseling and Planning Volume 17, Issue 2 2006 



ine whether changes in financial behavior would have 
occurred even without the program. Unfortunately, be¬ 
cause of resource constraints, this level of “rigor” is not 
feasible for most non-profit organizations. As researchers 
and funders push for more rigorous evaluations, it is im¬ 
portant that the profession continue to recognize the value 
of more traditional evaluation methods such as RPTs, pre- 
and post-tests, and qualitative surveys that collect best 
practices and success stories. The value of these types of 
methods should not be overlooked. As this study has 
shown, RPTs can still provide useful insight into the 
effectiveness of financial education. The self-reported, 
retrospective measures used in this study, though imperfect 
indicators of actual and anticipated changes in behavior, 
may still serve as good indicators of the program’s impact. 
These indicators reflect changes not only in participants’ 
level of knowledge but also in participants’ confidence in 
their skills and in their ability to shape their future behav¬ 
iors. If participants are feeling better about their financial 
situation at the end of the program, this is a positive reflec¬ 
tion of what the program is trying to achieve. 

From a program delivery standpoint, programs that focus 
on more basic and fundamental decision-making skills 
may give participants the confidence they need to take the 
first step towards behavior change. These types of pro¬ 
grams may not need to have numerous and complicated 
financial lessons. A few general lessons in basic financial 
education are likely to result in positive outcomes. 
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Endnotes 

'The advantages and disadvantages of using a retrospec¬ 
tive pre-test are discussed in the section on data collection. 
2 The incidences of negative changes were very few and 
treated as no improvement. 
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Appendix 

Let z*, a random variable signifying the latent financial 
effectiveness of an individual, be distributed with the 
cumulative distribution function G(z). Although z* is 
unobservable, we observe its ordered ratings, pre and post, 
which are defined as 

pre (or post) = 1 , if z* < Lj 

= 2, ifLi<z* <X 2 
= 3, if L 2 < z* < L 3 
= 4, ifz* >L 3 

where Li < X 2 < L 3 . For simplicity, assume pre _L post (it 
can also be shown that the implication does not change 
when the two are correlated). The probability of improve¬ 
ment, conditional on the financial effectiveness, can be 
shown as 

Pr( post - pre > 0 | pre = j) = 1 - g(A .) 

for j = 1, 2, 3, and 0 for j = 4. Since G(.) increases in j, 

5Pr (post-pre> 0) < ^ 
dpre 

In other words, the chances of improvement decline as pre¬ 
program status increases, other things being equal. 
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